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(57) Abstract 

The present invention relates to a method and apparatus for determining visually noticeable differences between two input images. A 
first contrast image is generated from the firet input image and a second contrast image is generate from the second input image. A contrast 
difference image is then formed from the two contrast images. Pixels in the contrast difference image are selected which differ in intensity 
from corresponding pixels in the first contrast image by a threshold value which is referenced to the intensity of the corresponding pixels 
in the first contrast image. The selected pixels may be applied to control the performance of television signal compression circuits so as to 
control the image fidelity. 
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Determining Visually Noticeable Differences Between Two Images. 

The present invention relates to determining visually noticeable differences 
between two images. The invention has particular application to the encoding 
and later decoding of an image in such fields as television signal generation, 
compression and transmission. When encoding an original image for 
television transmission and display at a receiver, it is important to know how 
well the system for transmitting and receiving will perfonm in reproducing the 
original image. The performance of the system is related to the visually 
noticeable differences between the original image and the displayed image. 

A number of models have already been proposed to predict whether a human 
observer will be able to discriminate between two images. One such visual 
discrimination model is known as the Sarnoff visual discrimination model. A 
visual discrimination model may take, as input, a pair of images and provide 
an output which is a map showing the probability, as a function of position on 
the images, that an observer would be able to detect differences between the 
images. The map is referred to as a JND map showing Just Noticeable 
Differences. This JND map can itself be presented as an image, with higher 
grey levels corresponding to higher probabilities of discrimination. 

In an encoding algorithm to encode picture information using quantisation 
methods, the image fidelity should be directly proportional to the quantisation 
sampling density. This assumption may be en-oneous with the result that 
encoding bits^ are squandered where they have little impact on the fidelity of 
reproduction i.pf an image. A : better method of controlling quantisation 
pararneters iS; to base control on a JND map to obtain either more unifomn 
' image fidelity for a fixed bit rate of encoding or better bit rates for a desired 
. . level of fideli^. ■ ' 

It is an object of the present invention to improve the identification of 
noticeable differences between two images. 
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According to the present invention there is now provided a method of 
identifying visually noticeable differences between a first input image and a 
second input image, the method comprising the steps of: generating a first, 
reference contrast image from the , first input image; .generating a second 
contrast image from the second input image; forming a contrast difference 
image from the first and second contrast images;, and selecting, from the 
contrast difference image, each pixel which has an intensity exceeding a 
thresholf^., determined by the^ intensity , of the corresponding pixel in the. 
reference contrast image. 

Further according to the present invention, there is iprovided^ apparatus for 
identifying visually noticeable differences between a first input image and a 
second input image, the apparatus comprising: a generator to generate a first, 
reference contrast image from the first input image and a • second contrast 
image from the second input image; a difference forming circuit to form a 
contrast difference image from the first and second contrast images; and a 
selector to selectrfromnhe^contrast ^f^fferc^Ftce irnage' 

intensity exceeding a threshold, detemnined by the intensity of the 
corresponding pixel in the referehc§^contrast image. ' ■ = - - ' 

The selected pixels which exceed the threshold value are indicative of the 
visually noticeable differences' between the first input image and the second 
input image. Where the second input image is a compressed form of the first 
input image, the selected pixels will indicate where noticeable differences 
occur between them as a result of the compression. tih^ ' cfegree of' 
compression can thus be increased in those areas of each' image' where 
visually noticeable differences do not appear. 

The invention will now be described, by way of example, with reference to the 
accompanying drawings in which: 
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Figure 1 shows in block diagram form a video signal transmission apparatus 
erribodytng the' present invention; 

Figure d shows a video sighal frame transmitted by the apparatLjs of Figure 1 ; 

Figure 3 shows a control crrcijit' included in the apparatus bf Figure 1 ; 

Figure 4 shows a block of p'ixels that are processed by the circuit of Ftgiiire 3; 

Figure 5 shows a block diagram of steps used in processing signals in the 
circuit of -Figur©:3ltO' provide a map of just noticeable differences between two 
image frames; ; . j '- : ./ivt.. 

Figure 6, shows, a graph of threshold values applicable to the process steps in 

Figuref|5; - r r L - 

■' ' ' , . ■ ■ ' ■ ■ , .--i f - - ' 

^'9^r%ilrftevys..a.;coriyen^^^ ' ■-^V^^- 

Figure 8 shows the apparatus of .Figure ;7 mpdifred according to the ^invention - 
to add a control circuit; 

Figure 8A .shows details of the control , circuit added intcjhe apparatus , of 
Figures;. _,,.^..r: - ■ - ■ . : = 

Figure 9 shows a further television signal encoder rnodified to incorporate the 
control circuit of Figure 8A; and 

Figure 10 shows a statistical multiplexer apparatus incorporating a plurality of 
the control circuits of Figure 8A. 

Figure 1 shows a video signal transmission apparatus including an analogue 
to digital converter 10 to receive an input analogue television signal 11 for 
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conversion to a digital signal. The input digital signal is encoded and 
compressed in an encoder 12 and applied to a transmission circuit 13. The 
transmission circuit 13 broadcasts the compressed signal in conventional 
manner for reception by one or more receiver circuits 14. The receiver circuit 
14 applies the received signal to a decoder 15 for decoding and supply to a 
display 16/ - . . 

Figure:2 shows a frame 10 included in the digital' signal output from the 
encoder 12. As ts^well understood in the digital television art, "the digital 
television signal includes a sequence of 'frahies 10. iach'franiecbrisists of a 
matrix.of pixels divided into M x N blocks of pixels. ' •? 

The processes of encoding, compression, transmission and display all 
introduce some degree of unwanted noise info'' thi^ picture information 
displayed by the television display 16. At the display 16, the noise may be 
more noticeable in some picture blocks of the image than in others. A control 
circuit 30 is incorporated into the encoder 12 to cohtrorthe compression of 
thei television signal so rasnto^ reduce the eifect of the dftiWanteS ndte^^^ 
control circuit 30 is shown in further detail in Figure 3; - ■ ^ - ' ' 

ln :Figuref3, the digital television -signal froriri "the analogue to digital cohvertef ' 
lOv^s^ applied to a compressidh drcCiit 31 for fcompression ahci supply 16 an 
output terminal 38 connected to the transnnission circuit 13/' The digital 
television signal is also supplied to a coiitriast image generatdr ^33 The 
generator 33 generates a contrast image from the pixels of eacff iranrie iand ' 
applies the contrast image to a differerice- circuit 35. ' ''' 

The compressed signal at the output terminal 38 is' supplied to ' a^' 
decompression circuit 34 where the signal is subject to decompression and 
the resulting decompressed signal is passed to a 1 - frarn^ stdrie 32. the 
output of the 1 - frame store 32 is applied to a second cohtr^st image 
generator 36. The difference circuit 35 receives the contrast images from the 
two contrast image generators and produces a contrast difference image. The 
contrast difference image is mapped in a JND Oust noticeable "difference) 
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map circuit 37. The map circuit 37 applies an output signal to control the 
compression circuit 31. ^ ■ 

The operation of the control circuit shown, in Figure 3 will now be described 
with .reference to Figures 2. 4, 5 and 6. Each frame of the: digital television 
signal applied to the contrast image generator 33 and the compression' circuit 
37 includes the blocks of pixels already mentioned and shown in Figure 2. In 
th^e contrast image generator 33, each of the pixels is processed in relation to 
a 3 x 3. neighbourhood. area as shown; in. Figure 4. Each pixel ag is included in 
a,3 .X 3 ,areg- extending from the pixel aM.ui to the pixel ai^-u+i. The irhage 
generator 33 generates a contrast value Cjj for each pixel Bij according to. the 
relation; 

. . , : - ,:Ci i p^JIOOj* Lmax^ Lmin / 
; : ■ ' ! .J Lmax + Lmin 

\A[here,; LfTf^x is the ma){in;iuni luminance value ^of the . pixels , in the 3x3 .\ 

^ T'f".)^ ;^*^^ gijnimuipslumirtanqQ value:. off the/pixelsr i . 
in the 3x3 neighbourhood-a?^a.^;::: : ; \ . , y . m . v 

Re|e[ring to fftgure 5. a signal -franrie J , pj^jhe d television signal is 
receivec^.,(3y..the:Contrast image generatpr>f^3,jn ^tep60 at the same time that : 
ajSigi7al frame is received by the ^pontrast J mage, generator 3.5 in step 51:- 
Tfie .signal Iraw ' 's converted in step 52 into a contrast Jmage I as already 
described.. ^tpove with reference. to Figure^;, 3.. The,, signal frame II; is a 
reconstruction of the television signal frarne. which precedes the sjgnai frame 
I. The signal frame II is derived by the decompression circuit 34 and supplied 
tq the 1 - frarne store 32. The signal frame 11. supplied, in step 51, is converted 
in step .53,intp,a contrast image M by the contrast image generator 36. The 
contrast image generator; 36 uses the same process as the contrast image 
generator 33 to .generate its contrast image. 

The rdifference. circuit 35 compares the contrast value of each pixel in the 
contrast irnage .1 with the contrast value of each pixel in the contrast image II. 
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The result is a contrast difference Image in which the contrast image 
differences for each frame are mapped in comparison with the preceding 
frame in the television signal. 

The contrast image I is applied, in step 54, to the JND map circuit 37; The 
contrast difference image is applied, in step 55, to the JND map circuit 37. 
The contrast image I is used as a reference for each pixel of the contrast 
difference image so as to select pixels from the contrast difference image in 
each frame as will be explained with reference to Figure 6. 

In Figure 6 there is shown a, threshold curve Ct plotted against a difference 
value Ac along one axis and a refer;ence value C along the other axis. Each 
axis has a logarithmic scale. : . 

The value Ac is the difference value of a pixel in the cpntrast difference image 
supplied by the difference circuit 35. The reference value C is the pixel value 
of the corresponding pixel in the reference contrast image supplied by the 
contrast image gener^ator, 33. ,lt w^^^^^ be ot)serv^d that the thresfeolid. curs^e Ct. - 
includes ^ constarit portion for which.AC apprpxirga^tes to 1 for; values of log 
between 0 and 3. For values of log Q in, excess. ^pf ,3, the curve Ct has-a-. 
gradient of 0.87. . , 



For each pixel where the value of log AC exceeds the threshold Ct,Ta control 
sighal output is suppiied to an output temriina 1 47 from the JND map circuit 37. 
The dohtrol signals are supplied to the compressipn circuit 31^to,ppntrol the^ 
compression of the television signal from the analogue to digital converter 10. ^ 
The Signal compression can thereby be varied in a selective, manner. 4o 
achieve a better image fidelity for a fixed bit rate of encoding in the 
compression circuit 3 1 or better bit rates for a desired level of ficjelity. 

Further details'' regarding the manner in \Miich the compression may be 
selectively varied will now be described with reference to Figures 7 and 8. In: 
Figure 7. a conventional video encoder comprises an input - buffer 70 to 
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receive a digital video signal. The buffer 70 applies sliced and reordered 
picture frames to a summer 71 The summer 71 has an output connected to a 
discrete cosine transfomi circuit 72 In which a discrete cosine transform is 
performed on the output from the summer 71. The transfonn circuit 72 is 
conhected'to a quantisation circuit 73, which feeds a variable length coder 74. 
The variable length coder 74 supplies an output buffer 75. The output from 
the buffer 75 is an MPEG compressed bit stream representing the digital 
video'.Signal applied to the input buffer 70. " 

A reverse quantisation circuit 76 receives an input from the quantisation 
circuit'73: The circuit 76 IS connected to feed a reverse transform circuit 77, 
which ■perfonTTiS^^a>revei'se- discrete cosine transformation. Each frame of the 
input digital video signal is decompressed by the circuits 76 ahd 77 and 
applied to a frame store 78. A fonvard motion estimation circuit 79 derives 
forward motion •estirtiatibrr signals by corhpkring each frame in the store 78 
with the frarr;ie^at the output from the buffer 70. 

A;fbrtward^rTibtion cbnipensatioll circiiit 80 receives the frsime stored in the M 
frame' '^stbre 78'^ and ihU WGiM^' W^a^6^f si^^s from 'th4 ' circuit 79. The 
motion compensatidri circuit- BO 'applies motion compensation to the video 
signals passing through the summer 71 . 

In Figure 8. the apparatus of Figure 7 is shown adapted to include a controj , 
circuit 82;^The control circuit 82 Vieceives at input 84 the decompressed and 
motion cdmpfeh^sated signal generated by the circuit 80. The circuit 82 also . 
receives at input '86 the picture informatioii from the buffer 70 in respect of the 
next sucbeedifig franrie. The circuit 82 has an olitput tenninal 88 connected to 
the%arisfdfrri circuit 72. 

Figure 8A shdw^ further detail of the control circuit 82. As seen in Figure 8A, 
the input temninal 84 is connected to a first contrast image generator 85 and 
the injDut tefrriinai 86 is corihected to a second contrast image generator 87. 
The two ebnfrast image generators 85 ahd 87 correspond respectively to the 
two' contrast imiige generators 33 and 36 of Figure 3. The contrast image 
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generators 85 and 87 generate contrast images in the same . manner as the 
corresponding contrast image generators 33 and 36 of Figure 3. The contrast 
images generated by the generators 85 and 87. are applied to a difference 
circuit 89, which corresponds, to the difference circuit 35 in Figure 3. The 
output from the . difference circuit 89 is applied -to a JND map^ circuit 90 
corresponding to the map circuit 37 of Figure 3. 

The elements of the control circuit 82^ operate in the same manner as the 
corresppnding elernents in the cpntrpi, circuit of Figure 3 to generate control 
signals ^t the output tenninal 88,; The cpntrol signals, are applied.tp control the 
transform circuit 72 such that.DCT values are sent when the control signal ■ 
exceeds the threshold Ct but are not sent when the control .signal js below the > 
threshold Ct. . r.,, ^ , . : . - . . 

In Figure 9, a second adaptation of the apparatus of Figure 7 is shown. In this 
second adaptation, the quantisation circuit ,73 repeives the .output from the 
transform, circuit 72 by way of a delayjCircuit 91. The control circuit 82 has its : 
inpAJt terminal 84 connected. Ig^a ^fi^me, .store iSt?, ; its > .input rteriTiinaL. 86 - . 
connected the input buffer TO and its output.terminal 88, connected to the r 
quantisation circuit 73 and the inverse quantisation^ circuit 76. A. quantisation;, 
circuit 94 receives the output from the transfomn circuit 72 and supplies a 
quantised, signal to an, invers^ quantisatiqn. circuit 96. .The ; output frpm the 
inverse quantisation circuit i^ subject to ari; invers;e DCT transforrriation in an 
inverse transform circuit 98, and is supplied to .the 

The cpntrol.circuit 82 generates a control signal on the terminal 88 te:yary the: 
step size of the quantisation applied by the quantisatipn. circuit 73. The 
threshold Ct determines the quantisation, parameters and hencevthe image v 
fidelity of the MPEG bitstream supplied from the output buffer. 75.- ' - 

In Figure 10. a series of encoders 1 10a to 11 On each receives a digital video 
signal at a respective input tenninal 111a to 11 In. Each of the encoders 110 
quantises and encodes the respective input digital signal for supply to a 
statistical multiplexer 112. Only two encoders are shown in Figure 10 but it 
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will be appreciated by those skilled in the art that multiple encoders may be 
linked to a common statistical multiplexer 112. The bit rate of each encoder is 
a function ' of the input digitaP signal and each encoder places a varying 
demand on the bit rate capacity of the statistical multiplexer 112: The bit rate 
capacity of the multiplexer 1-12 il shared between the encoders I f Oa to 110n 
in a known manner ' 

Cbhtrbl cii-ciJrts 114a to 114n are provided for controlling the encdders 110a 
to 110h: Each of the control circuits 114a to ri4n hais the same conflgui-ation 
and funclibii as the control-^ circuit sKown in Figure 3 and described above. 
The cohtrdi circuits 114a to 114n each cbhtror the quantisation of the " 
respective ehcdder'td' achieve a higher bit rate for a given image fidelity. The 
number of encoders 110, which share the bit rate capacity of the statistical 
multiplexer 112. may thus be increased. 

Whilst the invention has been "described in relation to the control of television 
signals compressed by -discrete cosine transform methods " and by ^ 
qu^ntisartioh'^methodsv^ it will be apparent %af the ih\fehtion may '^1^^ be 
applied' ta'the dbhtrbl' of television sign^ls'^ cdnfipre^^ by ^bthef- techniques 'rC - 
sUdh^ag wavelet compi-essior^!^' - • '^ ^^'^^ ^ ^ . ; , ' : . / 

it Will be seen from the above descfiptibn !that the ihventidn provides a ^ 
method 'and apparatus which isi abfe to identify areas 1n an imagi4 where ' 
visually noticeable differences* occur bet^A^eh an - original irhage ' ahd a 
reproduction of that image. The performance of the system that provides the 
reproductible feari be derived by means of the ihvbhtibn and adjustment hSade 
to the svst^m - to' enable the requisite image fidelity 'to be achieved. Thus; 
where' the systferri compresses the original image, the degree of cohnpression 
can be adjusted to reduce or eliminate the usually noticeable differences. 
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CLAIMS 

"1 A method of identifying visually noticeable differences between a first 
input image and a second input image, the nnethpd comprising the 
steps of: 

generating a first, reference contrast image from the first input 

image, 

generating a second contast image from the second input 

image; 

forming a contrast difference image frpnn the^ first ^gqd second 
contrast images; and 

selecting, from the contrast difference image, each pixel which 
has an intensity exceeding a threshold determined by the intensity of 
the corresponding pixel in th^ reference contrast image. . , . 

2. A method as claimed in claim 1, wherein the first and secprid input 
irnages each consist of a matrix of pixels y/ithin a television . signal 
frame. 

A r^sthod as claimed clainn 2, wherein th^ first and second contrast , 
rnriages are generated by. processing _each of the pixels in the 
^respectiye frarne to produpe ".a contrast .value Cy for. each pixel 
according to the relation; 

, Cii= 100 * Lmax^Lmin . ^, , ^ 

_ Lma>(+.Lmin . ^ .,v„^ - 

Where Lmax is the maximum luminance value of the pixels jp, the 3x3 
neighbourhood area adjoining the pjxel and LmiOr is. Jfie . minimum 
luminance value of the pixels in the 3 x 3 neighbourhood area. 

4. A method as claimed in claim 1, 2 or 3, including the further step of 
deriving the second input image from the first input irnage by first 
compressing and subsequently decompressing the first input image. 
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5. A method as claimed in claim 4, wherein the step of deriving the 
second input image from the first input image comprises compressing 
'the first input image by a discrete cosine transformation step and 
decompressing the compressed first input image by an inverse discrete 
cosine transformation step. 

6 ■ ' 'A method as clairhed in claim 4, wherein the step of deriving the 
second input image from the first input image comprises compressing 
' the firs^ input image by a digital quaritisatioh' step and decompressing 
the compressed first input image by an inverse' digital quantisation 
step. 

7. A method a'^' biairVid'd in claim '5 or 6, comprising the further step of 
controlling the degree of compression by reference to the pixels 
selecitea from the contrast difference image, so as to lirnit the number 
of visually noticeaBie differences between the first and second input 
images. 

8/' ''' Apparatus for identffyitig visually 'noticeable biffe re h^^ between a first 
input image'^nd'a s6cdhd input imaged the apparatus comprising: 
' ' a generator to generate a Yirst; r'e^ 

the first input image and a second contrast Image from the second 
input image; 

a difference forming cirbuit to form a contrast difference image 
from the first and second contrast images; and 

a selector to select from the contrast difference image each 
' pixel, -which has an intensity exceeding a threshold, detenmined by the 
ihteinsity of the corresponding pixel in the reference contrast image. 

9. Apparatus as claimed in claim 8, wherein the generator is adapted to 
generate the first arid second contrast images from first and second 
input images* each consisting of a matrix of pixels within a television 
• signWl frame.' 
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10. Apparatus as claimed in claim 9, wherein the generator is adapted to 
generate the first and second contrast images by processing each of 
the pixels in the respective frame to produce a contrast value for 
each pixel. ai J according tp the relation; 

Cij = 100 * Lmax - Lmin 

Lmax+Lmin . - 

yVhere Lniax js the maximum lurninance value pf,the pixej^in the 3x3 
neighbourhood area adjoining the pixel and , Lmin i,s. the , minimum 
luminance value of the pixels in the 3x3 neighbourhood area. 

11. Apparatus as claimed in claim 3, 9 on 10, further in eluding compression 
: and decompression circuits adapted to derive the secon^ input image 

from the first input image by first compressing and .subs^quently 
decompressing the first input image. 

12. Apparatus as claimed , in claim 11, wherein the compressipn, circuit 
comprises means, to compress the; flr$t input Jm age by disci;ete cosine 
transformation and the decompression circuit cornpnses means to 
decompress the compressed first input image by inverse discrete 
cosine transformation. 

13. Apparatus as claimed in claim 11, wherein the compression circuit 
comprises means to compress the first input image by digital 
quantisation and the decompression circuit comprises means to 
decompress the compressed first input image by inverse digital 
quantisation. 

14. Apparatus as claimed in claim 11 or 12, further comprising control 
means to control the compression in the compression circuit in 
response to the pixels selected from the contrast difference image, so 
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as to limit the number of visually noticeable differences between the 
first and second input images. 

15/ Signal encoding and multiplexing 'apparatus comprising a plurality of 
signal encoders for encoding and compressing a repetitive plurality of 
input digital television signals to produce a corresponding plurality of 
encoded, compressed television signals and a statistical multiplexer to 
multiplex the encoded television signals, wherein each of the signal 
encoders includes a control circuit to control the degree of 
' comffte'ssion apjali^d by the sighareric'oder, the control circuit being as 
" claimeci in^laiHiiW/ ' - ■ : .■• 

16. A method of identifying visually noticeable differences between a first 
■ input lmage;'the niefHod being substantially as herein before described 

■■ with ' refefencte \o^ 3, 8,"' 9, or 10 of the accompanying 

'drawings.* ^ '''' 

17. Apparatus for identifying visually noticeable differences between a first 

iHr:,r:|p,^Q^?^jj=y-^g:^gjPj^ second^ ifiput iftiagfe/ '%e ''^pp^r^ti/s' beinig 
- substkritially as hereiH bfefdre descn bed with "reference to*" Figure 1, 3, 
" ='8, Si?or1d-6f trteaiccohipanyin^ ' ' ' 
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